Overview of Deep Neural Networks

In recent years, new neural network models with deep architectures started to get more attention in the field of machine learning. These models contain larger number of layers (therefore ”deep”) than conventional multi-layered perceptron, which usually uses only two or three functional layers of neurons. To overcome the difficulties of training such complex networks, new learning algorithms have been proposed for these models. The two most prominent deep models are the Deep Belief Networks and Stacked Auto-Encoders, which currently present state-of-the-art results in many traditional pattern recognition benchmarks. This article summarizes the advantages of deep architectures and provides a concise overview of the deep network models, their properties and learning algorithms.